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A NOTE ON MINORITY GROUP TEST BIAS STUDIES 



Richard R, Re illy- 
Educational Testing Service 

Abstract 

Comparisons of majority and minority group regression lines for purposes 
of assessing test bias may, under certain conditions, be viewed as comparisons 
of conditional bivariate distributions • Where these conditions hold, findings 
should reveal parallel regression lines except for a special case. Given the 
conditions described, one implication is that even when the test is a parallel 
form of the criterion, lines with equal slopes but unequal intercepts should be 
found. 



A NOTE ON MINORITY GROUP TEST BIAS STUDIES 

Richard R. Reilly 
Educational Testing Service 

In recent years a fair number of studies have at1:einpted to assess the 
cultural bias in tests by conducting validity studies for minority and majority 
groups. In the academic context the criterion is usually grades, while in the 
industrial situation supervisor's ratings are often utilized. Typically, such 
studies compare a minority group regression line with a majority group regres- 
sion line. The procedure described by Gulliksen and Wilks (1950) inay then be 
used to test sequentially differences between (a) the standard errors of esti- 
mates, (b) the slopes, and (c) the intercepts. Another method might be te use 
the P distribution to test equality of slopes and intercepts (e.g., Cleary, 
1968). Failure to reject all null hypotheses would suggest that the two groups 
were drawn from the same bivariate normal population. 

It is the purpose of this note to suggest that if certain conditions hold, 
identifying groups for the purposes of predictive bias studies may result in a 
crude form of statistical conditioning, which makes it likely that regression 
lines with equal slopes but unequal intercepts will be found. 

Though most cultural bias studies have dealt with categorically defined 
groups, if we assume a continuous normally distributed variable underlying the 
categorizations, then it is possible to consider a trivariate normal distribu- 
tion of the following variables all expressed in standard score terms ( \x = 0 ; 
(T = 1 ): 

y^*, a test variable 

yg , a criterion variable 

y^ , a variable underlying, or highly correlated with, the group 
categorizations, which will be referred to as a sociocultural 
variable in this paper. 
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The density function can be written as: 
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where R is the inverse of the variance-covariance (or, in this case, correla- 
tion) matrix, 
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and Y is the random vector with elements V-^ , , Y-^ ^ 

If we now take the conditional bivariate density function of 
and y^ for some given value of y^ = , we find 
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where R* is now the inverse of the variance-covariance matrix V* . V* may- 
be written in terms of the elements of the original V matrix as 



V* = 



(1 - P^5) 



(Pl2 - P15P25) 



(p^2 - P15P25) (1 - P25) 
and the elements of Y* may be written as 



Y* = 



^1 - Pl5^5 
y2 - P25Y^ 
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in other words, we novr have a bivariate normal distribution with means p,;,Y^ 



and Pg^Y^ . 



We can now find the slope of on as follows: 

(Pl2 - Pijp23) 

v'^i" .(i--p^^) • ■ . 

Note that this slope, which is a conditional slope, will be the same no 
2r what value of is chosen. 

Next we find the intercept, 



(1 - " • 

Thus, although the slope will remain invariant for different values of 
it can be seen that the intercept will vary depending on the value of y^ chosen, 
except for the special case of p^^ = • 

Under certain conditions, then, predictive bias studies might be viewed as 
comparisons of conditional normal bivariate distributions. This conditioning 
may occur as a result of the identification of group categories which represent 

crude points on a third normally distributed variable, y, . 

5 

If one accepts the trivariate normal asstimptions, then it is already clear 
that the slopes will be the same for all groups and that intercepts will differ 
(except for the special case of p^^ = ^12^15^ • Correlations may differ within 
groups because of selection on the predictor, which may be more extreme for some 
groups than others. This does not affect the above argument, however, since the 
slopes, intercepts and standard errors should not be affected by selection. 
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The special case where p^^ = P^^gP-j^^ will result in equal intercepts no 
matter what values of are chosen. Three specific instances of this set 

of conditions are worth noting: 

(a) The case where is uncorrelated with both test and criterion 
(i.e., p^^ = = 0 ). 

(b) The case where the test and criterion are perfectly correlated 

pj^ = 1.0 ). 

(c) The case where y^ is more highly correlated with the test than 
with the criterion to the extent that the condition p^^ = P^g^ij 
holds . 

Although all three examples would result in equal regression lines on2y, 
cases (a) and (b) can be considered completely unbiased. Case (c), as Thorndike 
(1971) has pointed out, is "unfair" in terms of the group proportions which 
would be selected given a fixed cutting score on the predictor. 

Whether the lines differ and how "Chey differ will depend on the elements 
of and the values of y^ used to form the conditional bivariate distribu- 

tions. In general, however, it is safe to say the following aboub most recent 
studies in this area: 

(a) A positive correlation exists between test and criterion. 

(b) If the sociocultural variable is defined such that individuals 
generally recognized as more advantaged have higher scores, the 
sociocultural variable is positively correlated with both test and 
criterion. 

Consider now the situation which is most desirable (in terms of the bias 
problem) short of having a perfect test-criterion correlation. This would be 
a situation where the test is actually a parallel form of the criterion. That 
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is, except for uncorrelated errors of measurement the test is measiiring the same 
thing as the criterion* We have, therefore, a bivariate normal population in 
which , the test-criterion correlation, is actually a reliability coefficient. 

Here, clearly, p^^ = p^^ , since y^ and y^ are parallel and p^^ = p^^ 1^12 • 
The situation where all three co relations are equal is highly unlikely since this 
would mean the sociocultural verj.aile, y^ , was parallel with y^ and y^ • 
Hence, we can say with some generality that p-j^^ = p^^ < p^^ 

If we now find two conditional normal bivariate density functions for y^ 
and y^ by taki ig two points on y^ , say, Y^^^j and > where Y^(^) > 

^5(b) ' deduce the following: 

(1) The slopes of the regression of y^ on y^ will be the same for both 
conditional distributions since the covariance matrix V* remains 
invariant for all values of y^ • 

(2) The intercepts for the conditional distributions will be 

= — ^ ^ , for group A 

(1 - P15) 

and 

Kg = — ^ ^ , for group B 

(1 - P15) 

Since all correlations are positive and p^^ = p^^ , we find that > • 
That is, the intercept for the "high" sociocultural group is greater than 
the intercept for the "low" sociocultural group, or to put it another way, group 
B is consistently overpredicted if predictions are based on the group A regres- 
sion equation. It is also clear that as p^^ 1»0 "the difference in intercepts 
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becomes smaller. Thoiigh the above argument is given for the case where y.^^ 
and y^ are parallel test forms the same results will be obtained in the case 
where y^ is correlated with factors common -o y^ and y^ and uncorrelated 
with all other factors. It is worth noting that in this situation if the socio- 
cultural variable is ignored and the group A regression line is used, any cut- 
ting score on the predictor will yield the "correct" proportions of each group 
in terms of the percentages expected to perform at a corresponding level on 
the criterion. 

Although a number of studies have reported parallel regression lines (e.g., 
Cleary, I968; Temp, 1971) a recent report by Campbell, Pike^ and Plaugher (l9f>9) 
is particularly worth noting because the criterion employed was a carefully con- 
structed, objective, job-knowledge test. For seven of eight predictors studied 
the regression lines for minority and majority groups were parallel with lower 
intercepts for the culturally disadvantaged group. 

Further empirical evidence for this phenomenon is presented in Table 1* 

! 

! Insert Table 1 about here 

The "criterion" in this instance is a roughly parallel form of the "predictor." 
Both variables are separately timed reading comprehension sections of the Law 
School Admission Test (data were collected during a regular national administra- 
tion). Here, it' can be seen that althotigh the slopes are very nearly the same, 
the difference in intercepts is marked, with the lower intercept belonging to 
the regression line for Blacks. 

The specific conditions described in this paper are not the only way in 
which overprediction can occur, of course. Linn and Werts (1971 ) describe some 
more general possibilities in a recent paper. The argument presented here 
assumes that groups are defined by conditioning on the same bivariate 



distribution. Although it may well be that the groups are from quite different 
bivariate populations, it seems reasonable that when objective measures such as 
tests serve as both predictors and criteria, the argument will hold* 

Conclusions 

When conducting minority group bias studies investigators should recognize 
that in some cases they may be comparing two conditional bivariate distributions 
from the same general bivariate population. Under the conditions described when 
the sociocultural variable is correlated with factors common to the predictor 
and the criterion but uncorrelated with all other factors the finding of "over 
prediction" for the lower sociocultural group should be expected. 
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Footnote 

This statement is not completely general since it does not take into accourit 
cases where y.^^ is the y^ true score (or vice versa) or where y^ is the under- 
lying true score. I am indebted to Dr. Robert L. Linn for pointing this out. 
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Table 1 

Selected Statistics Computed Separately for Blacks and Whites^ 

-JL Xp Xc Sp So ^pc Slope Intercept 

Blacks 500 12.5 11.5 h.2 5.9 .6l =^7 1^.5 
Whites 500 l5.1^ ll^.8 1^.5 k.2 .62 5.9 



The criterion, Xc , is a separately timed reading comprehension 
section of the LSAT, as was the predictor Xp . 



